Markov Chain Monte Carlo Inference Research Articles

Bayesian inference of structured latent spaces from neural population activity with the orthogonal stochastic linear mixing model.

The brain produces diverse functions, from perceiving sounds to producing arm reaches, through the collective activity of populations of many neurons. Determining if and how the features of these exogenous variables (e.g., sound frequency, reach angle) are reflected in population neural activity is important for understanding how the brain operates. Often, high-dimensional neural population activity is confined to low-dimensional latent spaces. However, many current methods fail to extract latent spaces that are clearly structured by exogenous variables. This has contributed to a debate about whether or not brains should be thought of as dynamical systems or representational systems. Here, we developed a new latent process Bayesian regression framework, the orthogonal stochastic linear mixing model (OSLMM) which introduces an orthogonality constraint amongst time-varying mixture coefficients, and provide Markov chain Monte Carlo inference procedures. We demonstrate superior performance of OSLMM on latent trajectory recovery in synthetic experiments and show superior computational efficiency and prediction performance on several real-world benchmark data sets. We primarily focus on demonstrating the utility of OSLMM in two neural data sets: μECoG recordings from rat auditory cortex during presentation of pure tones and multi-single unit recordings form monkey motor cortex during complex arm reaching. We show that OSLMM achieves superior or comparable predictive accuracy of neural data and decoding of external variables (e.g., reach velocity). Most importantly, in both experimental contexts, we demonstrate that OSLMM latent trajectories directly reflect features of the sounds and reaches, demonstrating that neural dynamics are structured by neural representations. Together, these results demonstrate that OSLMM will be useful for the analysis of diverse, large-scale biological time-series datasets.

Estimating Posterior Sensitivities with Application to Structural Analysis of Bayesian Vector Autoregressions

– The inherent feature of Bayesian empirical analysis is the dependence of posterior inference on prior parameters, which researchers typically specify. However, quantifying the magnitude of this dependence remains difficult. This paper extends Infinitesimal Perturbation Analysis, widely used in classical simulation, to compute asymptotically unbiased and consistent sensitivities of posterior statistics with respect to prior parameters from Markov Chain Monte Carlo inference via Gibbs sampling. The method demonstrates the possibility of efficiently computing the complete set of prior sensitivities for a wide range of posterior statistics, alongside the estimation algorithm using Automatic Differentiation. The method’s application is exemplified in Bayesian Vector Autoregression analysis of fiscal policy in US macroeconomic time series data. The analysis assesses the sensitivities of posterior estimates, including the Impulse response functions and Forecast error variance decompositions, to prior parameters under common Minnesota shrinkage priors. The findings illuminate the significant and intricate influence of prior specification on the posterior distribution. This effect is particularly notable in crucial posterior statistics, such as the substantial absolute eigenvalue of the companion matrix, ultimately shaping the structural analysis.

Posterior Manifolds over Prior Parameter Regions: Beyond Pointwise Sensitivity Assessments for Posterior Statistics from MCMC Inference

Abstract Increases in the use of Bayesian inference in applied analysis, the complexity of estimated models, and the popularity of efficient Markov chain Monte Carlo (MCMC) inference under conjugate priors have led to more scrutiny regarding the specification of the parameters in prior distributions. Impact of prior parameter assumptions on posterior statistics is commonly investigated in terms of local or pointwise assessments, in the form of derivatives or more often multiple evaluations under a set of alternative prior parameter specifications. This paper expands upon these localized strategies and introduces a new approach based on the graph of posterior statistics over prior parameter regions (sensitivity manifolds) that offers additional measures and graphical assessments of prior parameter dependence. Estimation is based on multiple point evaluations with Gaussian processes, with efficient selection of evaluation points via active learning, and is further complemented with derivative information. The application introduces a strategy to assess prior parameter dependence in a multivariate demand model with a high dimensional prior parameter space, where complex prior-posterior dependence arises from model parameter constraints. The new measures uncover a considerable prior dependence beyond parameters suggested by theory, and reveal novel interactions between the prior parameters and the elasticities.

Sequential reversible jump MCMC for dynamic Bayesian neural networks

The challenge to automatically design machine learning-based models for datasets with varying dimensions (features) remains open, especially in the context of unstructured and noisy data. Bayesian neural networks employ Markov chain Monte Carlo (MCMC) and variational inference methods for training (sampling) model parameters. However, the progress of MCMC methods in deep learning has been slow due to high computational requirements and uninformative priors of model parameters. Reversible jump MCMC allows sampling of model parameters of variable lengths; hence, it has the potential to train Bayesian neural networks effectively. In this paper, we implement reversible jump MCMC sampling for training dynamic Bayesian neural networks that feature cascaded neural networks with dynamic hidden and input neurons. Our dynamic Bayesian neural network can model unstructured data of varying dimensions and provide uncertainty quantification in model predictions. We apply the methodology to selected regression and classification problems from the literature. The results show that our method provides an effective approach for the dynamic exploration of models while featuring uncertainty quantification that not only caters to model parameters but also extends to model topology. This opens up the road for uncertainty quantification in dynamic deep learning models for dynamic and unstructured data streams.

Reconstructing physical parameters from template gravitational wave spectra at LISA: first order phase transitions

A gravitational wave background from a first order phase transition in the early universe may be observable at millihertz gravitational wave (GW) detectors such as the Laser Interferometer Space Antenna (LISA). In this paper we introduce and test a method for investigating LISA's sensitivity to gravitational waves from a first order phase transition using parametrised templates as an approximation to a more complete physical model. The motivation for developing the method is to provide a less computationally intensive way to perform Markov Chain Monte Carlo (MCMC) inference on the thermodynamic parameters of a first order phase transition, or on generally computationally intensive models. Starting from a map between the physical parameters and the parameters of an empirical template, we first construct a prior on the empirical parameters that contains the necessary information about the physical parameters; we then use the inverse mapping to reconstruct approximate posteriors on the physical parameters from a fast MCMC on the empirical template. We test the method on a double broken power law approximation to spectra in the sound shell model. The reconstruction method substantially reduces the proposal evaluation time, and despite requiring some precomputing of the mapping, this method is still cost-effective overall. In two test cases, with signal-to-noise ∼ 40, the method recovers the physical parameters and the spectrum of the injected gravitational wave power spectrum to 95% confidence. In previous Fisher matrix analysis we found the phase boundary speed v w was expected to be the best constrained of the thermodynamic parameters. In this work, for an injected phase transition GW power spectrum with v w = 0.55, with a direct sample on the thermodynamic parameters we recover 0.630+0.17 -0.059 and for our reconstructed sample 0.646+0.098 -0.075.

Defining genomic epidemiology thresholds for common-source bacterial outbreaks: a modelling study

Epidemiological surveillance relies on microbial strain typing, which defines genomic relatedness among isolates to identify case clusters and their potential sources. Although predefined thresholds are often applied, known outbreak-specific features such as pathogen mutation rate and duration of source contamination are rarely considered. We aimed to develop a hypothesis-based model that estimates genetic distance thresholds and mutation rates for point-source single-strain food or environmental outbreaks. In this modelling study, we developed a forward model to simulate bacterial evolution at a specific mutation rate (μ) over a defined outbreak duration (D). From the distribution of genetic distances expected under the given outbreak parameters and sample isolation dates, we estimated a distance threshold beyond which isolates should not be considered as part of the outbreak. We embedded the model into a Markov Chain Monte Carlo inference framework to estimate the most probable mutation rate or time since source contamination, which are both often imprecisely documented. A simulation study validated the model over realistic durations and mutation rates. We then identified and analysed 16 published datasets of bacterial source-related outbreaks; datasets were included if they were from an identified foodborne outbreak and if whole-genome sequence data and collection dates for the described isolates were available. Analysis of simulated data validated the accuracy of our framework in both discriminating between outbreak and non-outbreak cases and estimating the parameters D and μ from outbreak data. Precision of estimation was much higher for high values of D and μ. Sensitivity of outbreak cases was always very high, and specificity in detecting non-outbreak cases was poor for low mutation rates. For 14 of the 16 outbreaks, the classification of isolates as being outbreak-related or sporadic is consistent with the original dataset. Four of these outbreaks included outliers, which were correctly classified as being beyond the threshold of exclusion estimated by our model, except for one isolate of outbreak 4. For two outbreaks, both foodborne Listeria monocytogenes, conclusions from our model were discordant with published results: in one outbreak two isolates were classified as outliers by our model and in another outbreak our algorithm separated food samples into one cluster and human samples into another, whereas the isolates were initially grouped together based on epidemiological and genetic evidence. Re-estimated values of the duration of outbreak or mutation rate were largely consistent with a priori defined values. However, in several cases the estimated values were higher and improved the fit with the observed genetic distance distribution, suggesting that early outbreak cases are sometimes missed. We propose here an evolutionary approach to the single-strain conundrum by estimating the genetic threshold and proposing the most probable cluster of cases for a given outbreak, as determined by its particular epidemiological and microbiological properties. This forward model, applicable to foodborne or environmental-source single point case clusters or outbreaks, is useful for epidemiological surveillance and may inform control measures. European Union Horizon 2020 Research and Innovation Programme.

Modelling intra-annual tree stem growth with a distributional regression approach for Gaussian process responses

Abstract High-resolution circumference dendrometers measure the irreversible growth and the reversible shrinking and swelling due to the water content of a tree stem. We propose a novel statistical method to decompose these measurements into a permanent and a temporary component, while explaining differences between the trees and years by covariates. Our model embeds Gaussian processes with parametric mean and covariance functions as response structures in a distributional regression framework with structured additive predictors. We discuss different mean and covariance functions, connections with other model classes, Markov chain Monte Carlo inference, and the efficiency of our sampling scheme.

Photometric Calibrations of M-dwarf Metallicity with Markov Chain Monte Carlo and Bayesian Inference

Knowledge of stellar atmospheric parameters (T eff, , [Fe/H]) of M dwarfs can be used to constrain both theoretical stellar models and Galactic chemical evolutionary models, and guide exoplanet searches, but their determination is difficult due to the complexity of the spectra of their cool atmospheres. In our ongoing effort to characterize M dwarfs, and in particular their chemical composition, we carried out multiband photometric calibrations of metallicity for early- and intermediate-type M dwarfs. The third Gaia data release provides high-precision astrometry and three-band photometry. This information, combined with the 2MASS and CatWISE2020 infrared photometric surveys and a sample of 4919 M dwarfs with metallicity values determined with high-resolution spectroscopy by The Cannon and APOGEE spectra, allowed us to study the effect of the metallicity in color–color and color–magnitude diagrams. We divided this sample into two subsamples: we used 1000 stars to train the calibrations with Bayesian statistics and Markov Chain Monte Carlo techniques, and the remaining 3919 stars to check the accuracy of the estimations. We derived several photometric calibrations of metallicity applicable to M dwarfs in the range of −0.45 ≤ [Fe/H] ≤ + 0.45 dex and spectral types down to M5.0 V that yield uncertainties down to the 0.10 dex level. Lastly, we compared our results with other photometric estimations published in the literature for an additional sample of 46 M dwarfs in wide binary systems with FGK-type primary stars and found a great predictive performance.

AE localization issues in heavy reinforced post-tensioned concrete beams

Damage localization is a relevant issue in structural health monitoring (SHM) of concrete structures. A greater level of insight into the structure can be so reached enabling more informed maintenance decisions and reducing operation and maintenance costs. The basic approach to acoustic emission localization, which is often referred to as the Time of Arrival (TOA) method, is based on detecting an AE source at a number of spatially-distributed sensors. However, in anisotropic media such as reinforced concrete, the basic assumptions for the TOA method become largely invalid. Different robust methods suitable to anisotropic media, incorporating an angular-dependent velocity term into the minimization process have been developed in the past. However, the wave-velocity profile is not always known. More recently a number of other authors have adopted various Bayesian approaches as part of AE localization strategies, including the use of a Markov chain Monte Carlo inference scheme as well as nonlinear Kalman filters. The use of a probabilistic approach in source location has become increasingly applied for AE source localization in complex large structures. In heavy reinforced post-tensioned concrete structures acoustic wave path can be significantly influenced by rebars as well as metallic post-tensioning ducts. Cracks opening can further influence acoustic propagation paths and greatly influence the reliability of AE source localization algorithms. In the present paper, different localization procedures have been developed and tested on post-tensioned concrete beams during loading and unloading cycles as well as on small homogeneous concrete blocks. In a highly emissive environment, significant difficulties have been reported also in event identification thus further reducing localization procedure reliability. EWGAE 35, Ljubljana, Slovenia, 13th – 16th Sep. www.ewg

A Global Lexical Database (GLED) for Computational Historical Linguistics

This work presents a lexical database with cognate annotation and phonological alignment for over 6,500 documented language varieties. The database includes per-family and global phylogenetic resources and offers a pre-computed global tree for language variety distance from normalized trees obtained with Bayesian Markov Chain Monte Carlo (MCMC) inference. Lexical data is provided in a single tabular file for convenience of usage, and resources are built adhering to best practices and state-of-the-art algorithms for historical linguistics. The database is a convenient source for research prototypes, method development, and analysis bootstrap. All resources are freely available for download for all interested researchers.

Flexible cloglog links for binomial regression models as an alternative for imbalanced medical data.

The complementary log-log link was originally introduced in 1922 to R. A. Fisher, long before the logit and probit links. While the last two links are symmetric, the complementary log-log link is an asymmetrical link without a parameter associated with it. Several asymmetrical links with an extra parameter were proposed in the literature over last few years to deal with imbalanced data in binomial regression (when one of the classes is much smaller than the other); however, these do not necessarily have the cloglog link as a special case, with the exception of the link based on the generalized extreme value distribution. In this paper, we introduce flexible cloglog links for modeling binomial regression models that include an extra parameter associated with the link that explains some unbalancing for binomial outcomes. For all cases, the cloglog is a special case or the reciprocal version loglog link is obtained. A Bayesian Markov chain Monte Carlo inference approach is developed. Simulations study to evaluate the performance of the proposed algorithm is conducted and prior sensitivity analysis for the extra parameter shows that a uniform prior is the most convenient for all models. Additionally, two applications in medical data (age at menarche and pulmonary infection) illustrate the advantages of the proposedmodels.

ToMExO: A probabilistic tree-structured model for cancer progression.

Identifying the interrelations among cancer driver genes and the patterns in which the driver genes get mutated is critical for understanding cancer. In this paper, we study cross-sectional data from cohorts of tumors to identify the cancer-type (or subtype) specific process in which the cancer driver genes accumulate critical mutations. We model this mutation accumulation process using a tree, where each node includes a driver gene or a set of driver genes. A mutation in each node enables its children to have a chance of mutating. This model simultaneously explains the mutual exclusivity patterns observed in mutations in specific cancer genes (by its nodes) and the temporal order of events (by its edges). We introduce a computationally efficient dynamic programming procedure for calculating the likelihood of our noisy datasets and use it to build our Markov Chain Monte Carlo (MCMC) inference algorithm, ToMExO. Together with a set of engineered MCMC moves, our fast likelihood calculations enable us to work with datasets with hundreds of genes and thousands of tumors, which cannot be dealt with using available cancer progression analysis methods. We demonstrate our method's performance on several synthetic datasets covering various scenarios for cancer progression dynamics. Then, a comparison against two state-of-the-art methods on a moderate-size biological dataset shows the merits of our algorithm in identifying significant and valid patterns. Finally, we present our analyses of several large biological datasets, including colorectal cancer, glioblastoma, and pancreatic cancer. In all the analyses, we validate the results using a set of method-independent metrics testing the causality and significance of the relations identified by ToMExO or competing methods.

Fatigue Detection of Pilots' Brain Through Brains Cognitive Map and Multilayer Latent Incremental Learning Model.

This work proposes a nonparametric prior induced deep sum-logarithmic-multinomial mixture (DSLMM) model to detect pilots' cognitive states through the developed brain power map. DSLMM uses multinormal distribution to infer the latent variable of each neuron in the first layer of the network. These latent variables obeyed a sum-logarithmic distribution that is backpropagated to its observation vector and the number of neurons in the next layer. Multinormal distribution is used to segment the extended observation vector to form a matrix associated with the width of the next layer. This work also proposes an adaptive topic-layer stochastic gradient Riemann (ATL-SGR) Markov chain Monte Carlo (MCMC) inference method to learn its global parameters without heuristic assumptions. The experimental results indicate that DSLMM can extract more probability distribution contained in the brain power map layer by layer, and achieve higher pilot cognition detection accuracy.

Model averaging for sparse seemingly unrelated regression using Bayesian networks among the errors

Multivariate Bayesian linear regression (MBLR) is a popular statistical tool with many applications in a variety of scientific fields. However, a shortcoming is potential model over-complexity, as the model assumes that all responses depend on the same covariates and that all errors are mutually pairwise correlated. The class of Bayesian seemingly unrelated regression (SUR) models generalizes the class of MBLR models by allowing for response-specific covariate sets. In a recent work it has been proposed to employ Gaussian graphical models for learning sparse SUR (SSUR) models with conditional independencies among the errors. The proposed SSUR model infers undirected edges among the errors, and the proposed Reversible Jump Markov Chain Monte Carlo (RJMCMC) inference algorithm relies on approximations of the marginal likelihoods. In this paper, we propose a new refined SSUR model that replaces the undirected graphs (Gaussian graphical models) by directed acyclic graphs (Gaussian Bayesian networks). Unlike the earlier proposed model, our new model is therefore able to learn some directed edges among the errors. And we derive a RJMCMC algorithm that does not require approximations of the marginal likelihoods. In particular, we present an algorithm for sampling covariance matrices that are coherent with a given directed acyclic graph. The proposed RJMCMC algorithm allows for exact Bayesian model averaging across both: the response-specific covariate sets and the directed acyclic graphs.

Probabilistic fatigue life analysis considering mean stress effects of fiber reinforced polymer (FRP) composites

In general, the S-N curve, which describes the relationship between fatigue life and stress level with a typical guarantee rate, is employed during the design of FRP structures. However, the effect of mean stress on fatigue life is typically ignored. In this paper, a revision term is proposed to consider the mean stress effects on the basis of the common S-N equation. The third material parameter of the revised S-N-R equation increased the difficulty to predict the probabilistic fatigue life with the traditional analysis method during the design of FRP structures. Hence, a failure probability evaluation method is proposed to obtain the probabilistic fatigue life with three material parameters by using Markov chain Monte Carlo (MCMC) and Bayesian inference. The results show the applicability and effectiveness of the method in evaluating failure probabilities.

Small-scale magnetic fields of the spectroscopic binary T Tauri stars V1878 Ori and V4046 Sgr

Aims. The goal of this study is to investigate the small-scale magnetic fields of the two spectroscopic binary T Tauri stars V1878 Ori and V4046 Sgr. This is done to complete the observational characterisation of the surface magnetic fields of these stars because only their large-scale magnetic fields have been studied with Zeeman Doppler imaging (ZDI) so far. Methods. To investigate the small-scale magnetic fields, the differential Zeeman intensification of near-infrared Ti I lines was investigated using high-resolution archival spectra obtained with the ESPaDOnS spectrograph at the CFHT. In order to study the binary components separately, the spectra were disentangled by considering observations taken at different orbital phases. The Zeeman-intensification analysis was performed based on detailed polarised radiative transfer calculations aided by the Markov chain Monte Carlo inference, treating magnetic field filling factors and other stellar parameters that could affect the spectra as free parameters. Results. The obtained average magnetic field strengths of the components of V1878 Ori are 1.33 ± 0.08 and 1.57 ± 0.09 kG, respectively. Previous ZDI studies of V1878 Ori recovered about 14 and 20% of this total magnetic field strength. For V4046 Sgr, the magnetic field strengths are 1.96 ± 0.18 and 1.83 ± 0.18 kG, respectively. In this case, about 12 and 9% of the total magnetic field strength was detected by ZDI. Conclusions. The small-scale magnetic field strengths obtained from Zeeman intensification are similar for the two components of each binary. This is in contrast to the large-scale magnetic fields obtained from ZDI investigations, performed using the same observations, which found that magnetic field strengths and topologies vary significantly in the components of the two binaries. While the large-scale field might look significantly different, the overall magnetic energy, primarily carried by the small-scale magnetic fields, appears to be quite similar. This indicates that the efficiency of the magnetic dynamo is comparable for the components of the two binaries.

Latent Set Models for Two-Mode Network Data

Two-mode networks are a natural representation for many kinds of relational data. These networks are bipartite graphs consisting of two distinct sets ("modes") of entities. For example, one can model multiple recipient email data as a two-mode network of (a) individuals and (b) the emails that they send or receive. In this work we present a statistical model for two-mode network data which posits that individuals belong to latent sets and that the members of a particular set tend to co-appear. We show how to infer these latent sets from observed data using a Markov chain Monte Carlo inference algorithm. We apply the model to the Enron email corpus, using it to discover interpretable latent structure as well as evaluating its predictive accuracy on a missing data task. Extensions to the model are discussed that incorporate additional side information such as the email's sender or text content, further improving the accuracy of the model.

BNNpriors: A library for Bayesian neural network inference with different prior distributions

Bayesian neural networks have shown great promise in many applications where calibrated uncertainty estimates are crucial and can often also lead to a higher predictive performance. However, it remains challenging to choose a good prior distribution over their weights. While isotropic Gaussian priors are often chosen in practice due to their simplicity, they do not reflect our true prior beliefs well and can lead to suboptimal performance. Our new library, BNNpriors, enables state-of-the-art Markov Chain Monte Carlo inference on Bayesian neural networks with a wide range of predefined priors, including heavy-tailed ones, hierarchical ones, and mixture priors. Moreover, it follows a modular approach that eases the design and implementation of new custom priors. It has facilitated foundational discoveries on the nature of the cold posterior effect in Bayesian neural networks and will hopefully catalyze future research as well as practical applications in this area.

Matter power spectrum emulator for f(R) modified gravity cosmologies

Testing a subset of viable cosmological models beyond General Relativity (GR), with implications for cosmic acceleration and the Dark Energy associated with it, is within the reach of Rubin Observatory Legacy Survey of Space and Time (LSST) and a part of its endeavor. Deviations from GR-w(z)CDM models can manifest in the growth rate of structure and lensing, as well as in screening effects on non-linear scales. We explore the constraining power of small-scale deviations predicted by the f(R) Hu-Sawicki Modified Gravity (MG) candidate, by emulating this model with COLA (COmoving Lagrangian Acceleration) simulations. We present the experimental design, data generation, and interpolation schemes in cosmological parameters and across redshifts for the emulation of the boost in the power spectra due to Modified Gravity effects. Three preliminary applications of the emulator highlight the sensitivity to cosmological parameters, Fisher forecasting and Markov Chain Monte Carlo inference for a fiducial cosmology. This emulator will play an important role for future cosmological analysis handling the formidable amount of data expected from Rubin Observatory LSST.

The metallicity of M dwarfs: photometric calibrations with Markov Chain Monte Carlo and Bayesian inference

The metallicity of M dwarfs: photometric calibrations with Markov Chain Monte Carlo and Bayesian inference